V Identification Entropy

Shannon (1948) has shown that a source (U, P, U) with output U satisfying Prob (U = u) = Pu, can be encoded in a prefix code C = {cu : u ∈ U} ⊂ {0, 1} * such that for the entropy H(P) = u∈U −pu log pu ≤ pu||cu|| ≤ H(P) + 1, where ||cu|| is the length of cu. We use a prefix code C for another purpose, namely noiseless identification , that is every user who wants to know whether a u (u ∈ U) of his interest is the actual source output or not can consider the RV C with C = cu = (cu 1 ,. .. , c u||cu||) and check whether C = (C1, C2,. . .) coincides with cu in the first, second etc. letter and stop when the first different letter occurs or when C = cu. Let LC(P, u) be the expected number of checkings, if code C is used. Our discovery is an identification entropy, namely the function HI (P) = 2 1 − u∈U P 2 u. We prove that LC(P, P) = u∈U Pu LC(P, u) ≥ HI (P) and thus also that L(P) = min C max u∈U LC(P, u) ≥ HI (P) and related upper bounds, which demonstrate the operational significance of identification entropy in noiseless source coding similar as Shan-non entropy does in noiseless data compression. Also other averages such as ¯ LC(P) = 1 |U | u∈U LC(P, u) are discussed in particular for Huffman codes where classically equivalent Huffman codes may now be different. We also show that prefix codes, where the codewords correspond to the leaves in a regular binary tree, are universally good for this average.